Method and apparatus for reconstruction of original images from modified images

ABSTRACT

An image processing method is disclosed, comprising comparing a first image and a modified second image, the modified second image comprising a copy of the first image in which at least one pixel value of one or more pixels have been modified, and based on the result of the comparison, generating first image reconstruction information which can be combined with the modified second image to reconstruct the first image. A corresponding image reconstruction method comprises combining the image reconstruction information with the modified second image to reconstruct the first image. Apparatus for implementing the methods is also disclosed.

TECHNICAL FIELD

The present invention relates to methods, apparatus and computerprograms for enabling an original image to be reconstructed from amodified copy of the image.

BACKGROUND ART

When creating multimedia content, some content creators may wish tomodify the original content in some way. For example, original contentsuch as images or video can be modified during editing by adding awatermark, changing privacy conflicting content (e.g. blurring out oflogos or faces), adding embellishments such as icons, writing ordrawings, applying filters, or adding interactive objects. When theoriginal content is modified to produce an edited image or video, theoriginal content will be permanently lost. If necessary, a copy of theoriginal content may be stored together with the edited image or video,to preserve the original content.

Reconstructing image is associated with filing unknown regions of animage, including removing the modified portion of the copy image andrecovering the removed portion of the original image. There is a needfor a method for efficiently reconstruction an original image from amodified copy image. The invention is made in this context.

DISCLOSURE OF INVENTION Technical Problem

An aspect of the present disclosure to provide a method and an apparatusfor efficient reconstruction of original images from modified images.

Solution to Problem

According to various embodiments of the present disclosure, a method forprocessing an image is provided. The method comprises: comparing a firstimage and a modified second image, the modified second image comprisinga copy of the first image in which at least one pixel value of at leastone replaced pixel has been modified; and based on the result of thecomparison, generating first image reconstruction information which canbe combined with the modified second image to reconstruct the firstimage.

According to various embodiments of the present disclosure, a method forreconstructing an image is provided. The method comprises: obtainingfirst image reconstruction information associated with a modified secondimage; and combining the first image reconstruction information with themodified second image to reconstruct a first image, the modified secondimage comprising a copy of the first image in which at least one pixelhas been replaced.

According to various embodiments of the present disclosure, anon-transitory computer readable storage medium adapted to storecomputer program instructions which, when executed, perform a methodaccording to any one of the preceding methods.

According to various embodiments of the present disclosure, an apparatusfor processing an image is provided. The apparatus comprises: at leastone processor; and memory adapted to store computer program instructionswhich, when executed by the at least one processors, cause the imageprocessing apparatus to compare a first image and a modified secondimage, the modified second image comprising a copy of the first image inwhich at least one pixels have been replaced, and based on the result ofthe comparison, generate image reconstruction information which can becombined with the modified second image to reconstruct the first image.

According to various embodiments of the present disclosure, an apparatusfor reconstructing an image is provided. The apparatus comprises: atleast one processor; and memory adapted to store computer programinstructions which, when executed by the at least one processors, causethe image reconstruction apparatus to obtain image reconstructioninformation associated with a modified second image, and combine theimage reconstruction information with the modified second image toreconstruct a first image, the modified second image comprising a copyof the first image in which at least one pixels have been replaced.

Advantageous Effects of Invention

Various embodiments of the present disclosure provide a method and anapparatus for efficient reconstruction of original images from modifiedimages.

BRIEF DESCRIPTION OF DRAWINGS

Embodiments of the present invention will now be described, by way ofexample only, with reference to the accompanying drawings, in which:

FIG. 1 illustrates a flowchart showing an image processing method,according to an embodiment of the present invention;

FIG. 2 illustrates a modified image obtained by combining an originalimage with additional content, according to an embodiment of the presentinvention;

FIG. 3 illustrates a method of generating image reconstructioninformation, according to an embodiment of the present invention;

FIG. 4 illustrates a method of reconstructing an original image usingthe image reconstruction information, according to an embodiment of thepresent invention;

FIG. 5 illustrates a flowchart showing an image reconstruction method,according to an embodiment of the present invention;

FIG. 6 illustrates a method of generating first image reconstructioninformation and second image reconstruction information for a firstmodified image and a second modified image, according to an embodimentof the present invention;

FIG. 7 illustrates a method of reconstructing the first modified imageand the original image using the first image reconstruction informationand the second image reconstruction information, according to anembodiment of the present invention;

FIG. 8 illustrates a system comprising an image processing apparatus andan image reconstruction apparatus, according to an embodiment of thepresent invention;

FIG. 9 illustrates a flowchart showing an image processing method,according to an embodiment of the present invention; and

FIG. 10 illustrates a flowchart showing an image reconstruction method,according to an embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

In the following detailed description, only certain exemplaryembodiments of the present invention have been shown and described,simply by way of illustration. As those skilled in the art wouldrealise, the described embodiments may be modified in various differentways, all without departing from the scope of the present invention.Accordingly, the drawings and description are to be regarded asillustrative in nature and not restrictive. Like reference numeralsdesignate like elements throughout the specification.

Referring now to FIG. 1, a flowchart showing an image processing methodis illustrated, according to an embodiment of the present invention.Depending on the embodiment, all steps in the method may be performed atthe same device, or different steps may be performed in differentdevices. Any of the steps illustrated in FIG. 1 may be performed insoftware or in hardware, depending on the particular embodiment. Whenone or more steps are performed in software, apparatus for performingthe method may include a processing unit comprising one or moreprocessors, and computer-readable memory having stored therein computerprogram instructions which, when executed by the processing unit,perform the respective method steps.

First, in step S101 comparing a first image is compared to a secondimage. The second image is a copy of the first image in which pixelvalues of one or more pixels have been modified, and is hereinafterreferred to as a modified second image. For example, the pixel values ofthe original image may have been modified by overlaying additionalcontent such as a watermark, logo, icon or other type of content on topof the original image, such that the original pixel information at thelocation of the additional content is lost in the modified second image.

FIG. 2 illustrates an example of a modified second image obtained bycombining an original image with additional content, according to anembodiment of the present invention. In the present embodiment themodified second image 220 is a frame of 360 degree video in which abroadcaster's logo 211 has been superimposed over content in the topright-hand corner of the screen. In other embodiments the techniquesdisclosed herein may be applied to different types of content other than360 degree video, such as two-dimensional video, scalable video or stillimages.

Continuing with reference to FIG. 1, in step S102, first imagereconstruction information is generated based on the result of thecomparison. The first image reconstruction information can be combinedwith the modified second image to reconstruct the first image. Forexample, as will be described in more detail later, the first imagereconstruction information may comprise residual pixel values calculatedas the difference between values of corresponding pixels in the firstimage and the modified second image, or may comprise the original valuesof pixels which have been replaced in the modified second image.

Then, in the present embodiment the first image reconstructioninformation is encrypted in step S103. Applying encryption to the firstimage reconstruction information can give the content creator controlover who is able to access the original content. Different encryptiontechniques may be used in different embodiments. For example, in stepS103 symmetric or asymmetric encryption may be used to encrypt the firstimage reconstruction information. In some embodiments encryption may beomitted, such that any party can access the first image reconstructioninformation and consequently reconstruct the original image.

When asymmetric encryption is used, only the private key holder will beable to decrypt the first image reconstruction information and proceedto reconstruct the original image. At the same time, asymmetricencryption can allow anyone to create content and protect the firstimage reconstruction information by applying encryption using the publickey.

In some embodiments additional content in a plurality of separate layersmay be combined with the original image. In such embodiments, separateimage reconstruction information may be generated for each layer ofadditional content, allowing intermediate modified images to bereconstructed by removing the additional content from specific layers.When image reconstruction information for a plurality of layers isgenerated, the image reconstruction information for each layer may beencrypted independently of the information for another layer. Forexample, the image reconstruction information for one layer ofadditional content may be encrypted using symmetric encryption, and theimage reconstruction information for a different layer of additionalcontent may be encrypted using asymmetric encryption.

Furthermore, when image reconstruction information for a plurality oflayers is generated, the image reconstruction information for differentlayers may be encrypted using different keys. For example, one set ofusers may be allowed to remove a top layer of additional content fromthe modified image, but only users who possess the decryption keys forall layers will be able to access the original content.

Through the use of image reconstruction information, embodiments of thepresent invention can enable the original content to be recoveredsubsequently after it has been modified. By applying encryption, asdescribed above in relation to step S103, it can be ensured that only aholder of the decoding key can access the original content.

In the present embodiment, once the first image reconstructioninformation has been generated and encrypted, in step S104 the encryptedfirst image reconstruction information is then stored as metadataassociated with the modified second image. For example, the first imagereconstruction information may be stored in the same file as themodified second image itself. Alternatively, the first imagereconstruction information may be stored separately to the modifiedsecond image. For example, the first image reconstruction informationmay be stored at a server and provided upon request to a user who wishesto reconstruct the original image.

Referring now to FIG. 3, a method of generating image reconstructioninformation is illustrated according to an embodiment of the presentinvention. In the present embodiment, first image reconstructioninformation is generated which comprises residual values 330. Eachresidual value 330 is associated with a pixel for which the originalpixel value has been replaced by a pixel of the additional content 310in the modified second image 320. A pixel in which the original valuehas been replaced by a pixel of the additional content 310 ishereinafter referred to as a “replaced pixel”.

As shown in FIG. 3, in the present embodiment an original image 300 iscombined with additional content 310 comprising a logo 311 in the upperright-hand corner of the image, similar to the example shown in FIG. 2.When combining the original image 300 and the additional content 310,each pixel of the additional content replaces the pixel at the samelocation in the original image 300. By combining the original image 300and the additional content 310 in this way, a modified second image 320is generated which comprises one or more replaced pixels.

Each residual value is generated by subtracting a value of a pixel inone of the first or second images from a value of the correspondingpixel in the other one of the first or second images 300, 320. Here, a“corresponding” pixel refers to a pixel at the same location in oneimage as the pixel currently being processed from the other image. Inthe present embodiment the value of a pixel in the original image 300 issubtracted from the value of the corresponding pixel at the samelocation in the modified second image 320.

When residual values are used as the first image reconstructioninformation 330, the format in which the residual values are stored mayvary according to the particular embodiment. For example, in someembodiments the residual values of any replaced pixels may be stored asmetadata included in the modified second image 320, together withcoordinates of the replaced pixels.

In other embodiments the residual values may be stored as a residualimage. In a residual image, a pixel at the same location as a replacedpixel in the modified second image 310 is used to store the residualvalue for that replaced pixel. All other pixels in the residual image,that is to say, all pixels which do not correspond to a replaced pixel,can be set to zero since the corresponding pixels have the same value inthe modified second image 320 and in the original first image 300. Sincemany of the residual values will be zero, the file size of the residualimage can be small relative to the original image file. Therefore theresidual image can be stored as metadata embedded in the modified secondimage 320 without significantly increasing the total file size.

A residual image may be generated by subtracting one of the first orsecond images 300, 320 from the other one of the first or second images300, 320. In some embodiments the subtraction calculation may beperformed for each pixel in the modified second image 320, resulting ina residual image with the same size and resolution as the first andsecond images 300, 320. The calculation will return a value of zero forany pixels which were not replaced in the modified second image 320,since these pixels will keep the same values as in the original firstimage 300.

Alternatively, a residual image may be generated by checking thecoordinates of the pixels included in the additional content 310, andonly performing calculations for the corresponding pixel locations inthe modified second image 320. The values of any remaining pixels in theresidual image may be automatically set to zero without performing acalculation for each pixel, since it may be assumed that thecorresponding pixels in the modified second image 320 and the originalfirst image 300 will have the same value as each other, and so theresult of the calculation would be zero. This approach can reduce theprocessing burden by avoiding having to perform unnecessarycalculations. However, in some embodiments it may be beneficial tocalculate a residual value for every pixel in the residual image. Forexample, if the modified second image 320 is compressed, the values ofsome pixels in the modified second image 320 may be different to thevalues of corresponding pixels in the first image 300 even though theyhave not been replaced by the additional content 310.

In some embodiments, the residual values may be stored at a higherprecision than the pixel values of the first and second images 300, 320.For example, in one embodiment each pixel value of the original firstimage 300 may be stored using 8 bits, and each residual value may bestored at a +2 bit precision relative to the original values, meaningthat each residual value may be stored using 10 bits. This approach canavoid any saturation occurring in bright or dark areas of the image.

As an alternative to using a residual image having the same size andresolution as the first and second images 300, 320, in some embodimentsthe residual image may take the form of a partial image having a smallersize than each of the first and second images 300, 320. A partialresidual image may be generated by subtracting part of the first imagefrom a corresponding part of the second image. For example, if theadditional content is restricted to a region of size N1×N2 pixels, thena partial residual image of resolution N1×N2 may be generated by onlysubtracting pixels within the N1×N2 region of one of the first andsecond images 300, 320 from the corresponding pixels of the other image300, 320. In this way, the partial residual image may have a lowerresolution than each of the first and second images.

When a partial residual image is generated, the first imagereconstruction information may also include information identifying aposition of the partial image within the first and second images. Insome embodiments this information could be omitted, for example if it isknown in advance that the additional content will always occupy the sameposition in the image. This may apply, for example, in the case of abroadcaster logo which is always displayed in the same screen positionthroughout all frames of a video.

In some embodiments in which partial residual images are used, theresidual image may be divided into a plurality of partial image regionsdistributed across the full area of the first and second images 300,320. The residual image may only include partial images for thosepartial image regions in which one or more pixels have been modified inthe second image 320. For example, each partial image region may beidentified by an index, and a partial image can be stored together withan identifier corresponding to the index of the partial image region towhich the partial image relates.

In the case of a video file comprising a sequence of frames, each framemay be divided into a plurality of partial image regions distributedacross the full area of the image. The plurality of partial imageregions may be equal in size, or may have different sizes. When pixelsof the video are modified, for example by combining the video withadditional content in the form of a logo or watermark, a plurality ofmodified frames are obtained. In any given modified frame, each partialimage region may or may not include modified pixels, that is, pixelsthat have been modified relative to an original version of the frame. Itcan be checked whether a particular partial image region containsmodified pixels by comparing pixels in the partial image region of themodified frame to corresponding pixels in the original version of theframe.

The partial image regions may be identified in different ways, forexample using the coordinates of one corner of the region, or thecoordinates of the centre of the region, or by assigning a unique indexto each region. In the present embodiment each partial image region isidentified using the coordinates of the pixel at the top left corner ofthe region. For example, a partial image region with coordinates 0,0 isthe region at the top left of the frame. Supposing that the region at0,0 contains modified pixels during the first 100 frames of the video, apartial residual video of 100 frames can be generated for the region at0,0. The partial residual video comprises a partial residual image foreach one of the first 100 frames of the video.

If the same region again contains modified pixels at frames 1000 to1100, a new sequence of partial residual images can be added to thepartial residual video with time stamps from 1000 to 1100, indicatingthat these partial residual images should be combined with therespective modified frames at time stamps 1000 to 1100 in the modifiedvideo. In this way the partial residual video of only 200 frames isrequired for 1100 frames of modified video, since the region in questiondoes not contain any modified pixels between frames 101 and 999. Thisapproach provides an efficient way of storing image reconstructioninformation for a video file.

As an alternative, in some embodiments the partial residual video may beextended by adding partial residual images with zero content betweenframes 101 to 999. As a further alternative, the new sequence of partialresidual images with time stamps from 1000 to 1100 can be stored as anew partial residual video, rather than as a continuation of the partialresidual video for frames 1 to 100.

In the embodiment shown in FIG. 3, the first image reconstructioninformation comprises residual values. In other embodiments the firstimage reconstruction information may be provided in a different form.For example, in some embodiments the first image reconstructioninformation may comprise one or more original pixel values eachassociated with a replaced pixel. Each original pixel value contains thevalue of the corresponding pixel in the first image 300. When originalpixel values are provided as the first image reconstruction information,the original image can be reconstructed by replacing the correspondingpixels in the modified second image with the original pixel values.

Referring now to FIG. 4, a method of reconstructing an original imageusing the image reconstruction information generated in FIG. 3 isillustrated, according to an embodiment of the present invention. Thefirst image reconstruction information is combined with the modifiedsecond image 320 by summing the one or more residual values with one ormore corresponding pixels in the modified second image 320, to determinevalues of said one or more corresponding pixels in the first image 300.In this way, a copy 400 of the original image 300 can be obtained. Forexample, when the first image reconstruction information comprises aresidual image, the original image can be reconstructed by summing theresidual image and the modified second image.

Here, ‘summing’ may involve either adding or subtracting, depending onwhether the residual is obtained by subtracting the modified image fromthe original image (or vice versa), or by adding the original image andthe modified image. For example, a residual value for the nth pixel, Rn,can be defined as:R _(n) =O _(n) −M _(n)  (1)or as:R _(n) =O _(n) +M _(n)  (2)

where O_(n) is the original value of the nth pixel in the originalimage, and M_(n) is the modified value of the n^(th) pixel in themodified image. In the case of equation (1) above, the original imagecan be reconstructed by calculating the original pixel value as follows:O _(n) =R _(n) +M _(n)  (3)

whereas in the case of equation (2) above, the original image can bereconstructed by calculating the original pixel value as follows:O _(n) =R _(n) −M _(n)  (4)

Referring now to FIG. 5, a flowchart showing an image reconstructionmethod is illustrated, according to an embodiment of the presentinvention. Depending on the embodiment, all steps in the method may beperformed at the same device, or different steps may be performed indifferent devices. Any of the steps illustrated in FIG. 5 may beperformed in software or in hardware, depending on the particularembodiment. When one or more steps are performed in software, apparatusfor performing the method may include a processing unit comprising oneor more processors, and computer-readable memory having stored thereincomputer program instructions which, when executed by the processingunit, perform the respective method steps.

The method starts in step S501 by obtaining first image reconstructioninformation associated with a modified second image. For example, instep S501 the first image reconstruction information may be retrievedfrom storage or may be received through a broadcast stream. When thefirst image reconstruction information is embedded as metadata in themodified second image, the first image reconstruction information andthe modified second image may be obtained at the same time in step S501.

In the present embodiment, the method as shown in FIG. 5 is configuredto reconstruct an image based on first image reconstruction informationthat was generated using a method similar to the one shown in FIG. 1, inwhich the first image reconstruction information is encrypted. As such,the method of FIG. 5 starts by decrypting the first image reconstructioninformation in step S502. In embodiments in which the first imagereconstruction information is not encrypted, step S502 may be omitted.

Then, in step S503 the first image reconstruction information 330 iscombined with the modified second image 320 in order to reconstruct thefirst image 400. The process by which the first image reconstructioninformation 330 is combined with the modified second image 320 may varyaccording to the format in which the first image reconstructioninformation 330 is provided. For example, in the embodiment describedabove with reference to FIG. 4, the first image reconstructioninformation 330 is combined with the modified second image 320 bysumming the one or more residual values with one or more correspondingpixels in the modified second image 320, to determine the original pixelvalues of the first image 300. Alternatively, if the first imagereconstruction information 330 contains the original pixel values of thefirst image 300, the first image reconstruction information 330 and themodified second image 320 can be combined by replacing pixels of themodified second image 320 with the corresponding original pixel valuesfrom the first image reconstruction information 330.

In some embodiments, additional content in a plurality of separatelayers may be combined with the original image. In such embodiments,separate image reconstruction information may be generated for eachlayer of additional content, allowing intermediate modified images to bereconstructed by removing the additional content from specific layers.

Referring now to FIG. 6, a method of generating first imagereconstruction information and second image reconstruction informationfor first and second modified images is illustrated, according to anembodiment of the present invention. Although two layers of additionalcontent are illustrated in FIG. 6, it will be understood that theprinciple can be extended to any number of layers.

In the embodiment of FIG. 6, an original image 600 is combined withfirst additional content 611 in order to generate a first modified image621, using a similar method as described above in relation to FIG. 3.Then, the first modified image 621 is combined with second additionalcontent 612 in order to generate a second modified image 622, againusing a similar method as described above in relation to FIG. 3.Depending on the relative positions of the first additional content 611and the second additional content 612, one or more pixels of the firstadditional content 611 may be replaced by the second additional content612. The second additional content 612 can therefore be considered asoccupying a higher layer than the first additional content 611, which inturn can be considered as occupying a higher layer than the originalimage 600.

As shown in FIG. 6, first image reconstruction information 631 andsecond image reconstruction information 632 is generated. The firstimage reconstruction information 631 can be combined with the firstmodified image 621 to reconstruct the original image 600. The firstimage reconstruction information 631 can be generated using a method asdescribed above in relation in FIG. 1, by comparing the first modifiedimage 621 to the original image 600. The second image reconstructioninformation 632 can be combined with the second modified image 622 toreconstruct the first modified image 621. The second imagereconstruction information 632 can also be generated using a method asdescribed above in relation in FIG. 1, by comparing the second modifiedimage 622 to the first modified image 621.

In the present embodiment, the first image reconstruction information631 is generated by subtracting a pixel value of the original image 600from a corresponding replaced pixel in the first modified image 621, andthe second image reconstruction information 632 is generated bysubtracting a pixel value of the first modified image 621 from acorresponding replaced pixel in the second modified image 622. Here, a“replaced pixel” in the second modified image 622 refers to a pixel ofthe first modified image 621 which has been replaced by the secondadditional content 612. In this way, the first image reconstructioninformation 631 and the second image reconstruction information 632 areboth generated as residual values.

In another embodiment, one or both of the first image reconstructioninformation 631 and the second image reconstruction information 632 maybe generated in a different format. For example, in one embodiment thefirst image reconstruction information 631 may comprise residual valuescalculated as the difference between corresponding pixels in the firstmodified image 621 and the original image 600, and the second imagereconstruction information 632 may comprise original pixel values fromthe first modified image 621 for any replaced pixels in the secondmodified image 622. As described above, the image reconstructioninformation for each layer may be encrypted independently of theinformation for another layer, and may be encrypted using differentkeys.

Referring now to FIG. 7, a method of reconstructing the first modifiedimage and the original image using the first and second imagereconstruction information generated in FIG. 6 is illustrated, accordingto an embodiment of the present invention. The method starts bycombining the second image reconstruction information 612 with thesecond modified image 622 in order to reconstruct the first modifiedimage 621. As described above in relation to step S503 of FIG. 5, theprocess by which the second image reconstruction information 612 iscombined with the second modified image 622 may vary according to theformat in which the second image reconstruction information 612 isprovided.

As with the method illustrated in FIG. 4, the first image reconstructioninformation 611 can then be combined with the first modified image 621by summing the one or more residual values with one or morecorresponding pixels in the first modified image 621, to determinevalues of said one or more corresponding pixels in the original image600. In this way, a copy 700 of the original image 600 can bereconstructed. If either of the first image reconstruction information611 and the second image reconstruction information 612 is encrypted,then the information may be decrypted using the appropriate decryptionkey before combining the decrypted information with the respective firstor second modified image 621, 622. As described above, different keysmay be used to decrypt the first image reconstruction information 611and the second image reconstruction information 612.

Although only two layers are described in the present embodiment, itwill be understood that the principles described here in relation toFIG. 7 may be extended to any arbitrary number of layers, to allow theintermediate modified images and the original image to be reconstructed.

Referring now to FIG. 8, a system comprising an image processingapparatus and an image reconstruction apparatus is illustrated,according to an embodiment of the present invention. In the presentembodiment the system comprises an image processing apparatus 800 forgenerating image reconstruction information, and comprises a separateimage reconstruction apparatus 900 for reconstructing an original imagebased on a modified image and the image reconstruction informationgenerated by the image processing apparatus 800. In other embodimentssome or all elements of the image processing apparatus 800 and the imagereconstruction apparatus 900 may be embodied in a single physicaldevice.

The image processing apparatus 800 comprises an image processing unit810 comprising one or more processors 811, and memory 812 adapted tostore computer program instructions. Although the image processing unit810 is implemented in software in the present embodiment, in otherembodiments the image processing unit 810 may be implemented inhardware, for example using a field programmable gate array (FPGA) chip.

When executed by the one or more processors 811 of the image processingunit 810, the computer program instructions stored in the memory 812cause the image processing apparatus 800 to compare an original imageand a modified image. Based on the result of the comparison, the imageprocessing apparatus 800 generates image reconstruction informationwhich can be combined with the modified second image to reconstruct theoriginal image. The computer program instructions stored in the memory812 can be configured to cause the image processing apparatus 800 toperform any of the methods disclosed herein for generating the imagereconstruction information.

Additionally, in the present embodiment the image processing apparatus800 further comprises an encryption unit 820 for encrypting the imagereconstruction information. The encryption unit 820 may, for example,comprise a hardware encryption unit. In another embodiment, theencryption unit 820 may be implemented in software in the form ofadditional computer program instructions stored in the memory 812.Furthermore, in some embodiments encryption may not be applied, and theencryption unit 820 may be omitted.

The image reconstruction apparatus 900 comprises an image reconstructionunit 910 comprising one or more processors 911, and memory 912 adaptedto store computer program instructions. Although the imagereconstruction unit 910 is implemented in software in the presentembodiment, in other embodiments the image reconstruction unit 910 maybe implemented in hardware, for example using a field programmable gatearray (FPGA) chip.

When executed by the one or more processors 911 of the imagereconstruction unit 910, the computer program instructions stored in thememory 912 cause the image reconstruction apparatus 900 to obtain imagereconstruction information associated with a modified image, and combinethe image reconstruction information with the modified image toreconstruct a copy of the original image. The computer programinstructions stored in the memory 912 can be configured to cause theimage reconstruction apparatus 900 to perform any of the methodsdisclosed herein for reconstructing an image based on imagereconstruction information.

Additionally, in the present embodiment the image reconstructionapparatus 900 further comprises a decryption unit 920 for encrypting theimage reconstruction information. The decryption unit 920 may, forexample, comprise a hardware encryption unit. In another embodiment, thedecryption unit 920 may be implemented in software in the form ofadditional computer program instructions stored in the memory 912.Furthermore, in some embodiments encryption may not be applied, and thedecryption unit 920 may be omitted.

FIG. 9 illustrates a flowchart showing an image processing method,according to an embodiment of the present invention.

Referring FIG. 9, in step 901, the method comprises comparing a firstimage and a modified second image, the modified second image comprisinga copy of the first image in which at least one pixel value of at leastone replaced pixel has been modified.

In step 903, the method comprises generating first image reconstructioninformation which can be combined with the modified second image toreconstruct the first image based on the result of the comparison.Preferably, wherein the first image reconstruction information comprisesat least one residual value each associated with one of the at least onereplaced pixel, and wherein each of the at least one residual values isgenerated by subtracting a pixel value of the at least one replacedpixel in one of the first image or the second image from a pixel valueof the at least one replaced pixel in the other one of the first imageor the second image. Preferably, wherein the first image reconstructioninformation comprises a residual image generated by subtracting one ofthe first image or the second image from the other one of the firstimage or the second image. Preferably, wherein the residual imagecomprises a partial image having a smaller size than each of the firstimage and the second image, wherein the partial image is generated bysubtracting a part of the first image from a corresponding part of thesecond image. Preferably, wherein the first image reconstructioninformation includes information identifying a position of the partialimage within the first image and the second image. Preferably, whereinthe residual image is divided into a plurality of partial image regionsdistributed across the full area of the first image and second image,and wherein the residual image comprises at least one partial image eachassociated with a corresponding one of the partial image regions inwhich at least one pixel has been modified in the second image.Preferably, wherein the first image reconstruction information comprisesat least one original pixel values each associated with one of the atleast one replaced pixels, each original pixel value being a value ofsaid replaced pixel in the first image.

According to various embodiments of the present disclosure, the methodmay further comprises: encrypting the first image reconstructioninformation. According to various embodiments of the present disclosure,wherein the first image and the modified second image correspond to atleast one sequence of video frames, and the method further comprises:for each of the at least one sequence of video frames, checking whethereach partial image region in the respective modified second image forthe video frames contains at least one modified pixel; in response to adetermination that one of the partial image regions contains at leastone modified pixel, generating respective first image reconstructioninformation for the video frames; and storing the respective first imagereconstruction information as a frame of a partial residual videocorresponding to the one of the partial image regions, wherein eachframe of the partial residual video is associated with a time stamp foridentifying the at least one sequence of the video frames to which theframe of the partial residual video relates.

According to various embodiments of the present disclosure, the methodfurther comprises: comparing the second image to a modified third image,the modified third image comprising a copy of the second image in whichat least one pixels have been replaced; and based on the result of thecomparison, generating second image reconstruction information which canbe combined with the modified third image to reconstruct the secondimage. According to various embodiments of the present disclosure, themethod further comprises: encrypting the second image reconstructioninformation, wherein an encryption key used to encrypt the first imagereconstruction information is different to an encryption key used toencrypt the second image reconstruction information.

According to various embodiments of the present disclosure, anon-transitory computer readable storage medium adapted to storecomputer program instructions which, when executed, perform a methodaccording to any one of the preceding methods.

According to various embodiments of the present disclosure, an apparatusfor processing an image is provided. The apparatus comprises: at leastone processor; and memory adapted to store computer program instructionswhich, when executed by the at least one processors, cause the imageprocessing apparatus to compare a first image and a modified secondimage, the modified second image comprising a copy of the first image inwhich at least one pixels have been replaced, and based on the result ofthe comparison, generate image reconstruction information which can becombined with the modified second image to reconstruct the first image.

FIG. 10 illustrates a flowchart showing an image reconstruction method,according to an embodiment of the present invention.

Referring FIG. 10, in step 1001, the method comprises obtaining firstimage reconstruction information associated with a modified secondimage. Preferably, wherein the first image reconstruction informationcomprises at least one residual value each associated with one of the atleast one replaced pixel, and wherein combining the first imagereconstruction information with the modified second image comprisessumming the at least one residual value with at least one correspondingpixel in the modified second image, to determine values of the at leastone corresponding pixel in the first image.

In step 1003, the method comprises combining the first imagereconstruction information with the modified second image to reconstructa first image, the modified second image comprising a copy of the firstimage in which at least one pixel has been replaced. Preferably, whereinthe first image reconstruction information comprises a residual imagegenerated by subtracting one of the first image or the second image fromthe other one of the first image or the second image, and whereincombining the first image reconstruction information with the modifiedsecond image comprises summing the residual image and the modifiedsecond image. Preferably, wherein the first image reconstructioninformation comprises at least one original pixel value each associatedwith one of the at least one replaced pixel, each original pixel valuebeing a value of said replaced pixel in the first image, and whereincombining the first image reconstruction information with the modifiedsecond image comprises replacing pixel values of at least one pixel inthe modified second image with the corresponding original pixel valuesfrom the first image reconstruction information.

According to various embodiments of the present disclosure, the methodfurther comprises: decrypting the first image reconstructioninformation. According to various embodiments of the present disclosure,wherein prior to reconstructing the first image the method furthercomprises: obtaining second image reconstruction information associatedwith a modified third image, the modified third image comprising a copyof the second image in which at least one pixel has been replaced; andcombining the second image reconstruction information with the modifiedthird image to reconstruct the second image. According to variousembodiments of the present disclosure, the method further comprises:decrypting the second image reconstruction information, wherein anencryption key used to decrypt the first image reconstructioninformation is different to an encryption key used to decrypt the secondimage reconstruction information.

According to various embodiments of the present disclosure, anon-transitory computer readable storage medium adapted to storecomputer program instructions which, when executed, perform a methodaccording to any one of the preceding methods.

According to various embodiments of the present disclosure, an apparatusfor reconstructing an image is provided. The apparatus comprises: atleast one processor; and memory adapted to store computer programinstructions which, when executed by the at least one processors, causethe image reconstruction apparatus to obtain image reconstructioninformation associated with a modified second image, and combine theimage reconstruction information with the modified second image toreconstruct a first image, the modified second image comprising a copyof the first image in which at least one pixels have been replaced.

Embodiments of the present invention have been described in whichinformation is generated that can be used to reconstruct an originalimage, from a copy of the image in which one or more pixels have beenmodified, for example by replacing the pixels with additional content.In some embodiments, the image reconstruction information may beencrypted. This approach allows users to modify the original content ina similar manner as in conventional content editing techniques, whilstensuring that the original content can still be recovered if desired.The file size of the image reconstruction information can besignificantly less than the file size of the original content.Therefore, less storage space may be required to store the modifiedimage and the image reconstruction information, in comparison to theamount of storage space that would be required to store both themodified image and the original image. Also, since the format of themodified image itself is not changed, the resulting image file can beunderstood by existing apparatus, for example a multimedia player,thereby providing backwards compatibility.

Whilst certain embodiments of the invention have been described hereinwith reference to the drawings, it will be understood that manyvariations and modifications will be possible without departing from thescope of the invention as defined in the accompanying claims.

The invention claimed is:
 1. A method for processing an image, themethod comprising: comparing a first image and a modified second image,the modified second image comprising a copy of the first image in whichat least one pixel value of at least one replaced pixel has beenmodified; and based on the result of the comparison, generating separateimage reconstruction information for each layer of additional contentwhich can be combined with the modified second image to reconstruct thefirst image.
 2. The method of claim 1, further comprising: encryptingthe separate image reconstruction information for each layer of theadditional content.
 3. The method of claim 1, wherein the separate imagereconstruction information for each layer of the additional contentcomprises at least one residual value each associated with one of the atleast one replaced pixel, and wherein each of the at least one residualvalue is generated by subtracting a pixel value of the at least onereplaced pixel in one of the first image or the second image from apixel value of the at least one replaced pixel in the other one of thefirst image or the second image.
 4. The method of claim 3, wherein theseparate image reconstruction information for each layer of theadditional content comprises a residual image generated by subtractingone of the first image or the second image from the other one of thefirst image or the second image.
 5. The method of claim 4, wherein theresidual image comprises a partial image having a smaller size than eachof the first image and the second image, and wherein the partial imageis generated by subtracting a part of the first image from acorresponding part of the second image.
 6. The method of claim 5,wherein the separate image reconstruction information for each layer ofthe additional content includes information identifying a position ofthe partial image within the first image and the second image.
 7. Themethod of claim 5, wherein the residual image is divided into aplurality of partial image regions distributed across the full area ofthe first image and second image, and wherein the residual imagecomprises at least one partial image each associated with acorresponding one of the partial image regions in which at least onepixel has been modified in the second image.
 8. The method of claim 2,wherein the encrypting of the separate image reconstruction informationfor each layer of the additional content comprises encrypting the imagereconstruction information for each layer independently from theinformation for another layer.
 9. The method of claim 1, wherein theadditional content includes first additional content and secondadditional content occupying a higher layer than the first additionalcontent, and wherein the separate image reconstruction information foreach layer includes: a first image reconstruction information used todelete the first additional content, and a second image reconstructioninformation used to delete the second additional content.
 10. A methodfor reconstructing an image, the method comprising: obtaining separateimage reconstruction information for each layer of additional contentassociated with a modified second image; and combining the separateimage reconstruction information for each layer of the additionalcontent with the modified second image to reconstruct a first image, themodified second image comprising a copy of the first image in which atleast one pixel has been replaced.
 11. The method of claim 10, furthercomprising: decrypting the separate image reconstruction information foreach layer of the additional content.
 12. The method of claim 10,wherein the separate image reconstruction information for each layer ofthe additional content comprises at least one original pixel value eachassociated with one of the at least one replaced pixel, each originalpixel value being a value of the replaced pixel in the first image, andwherein combining the separate image reconstruction information for eachlayer of the additional content with the modified second image comprisesreplacing pixel values of at least one pixel in the modified secondimage with the corresponding original pixel values from the separateimage reconstruction information for each layer of the additionalcontent.
 13. An apparatus for processing an image, the apparatuscomprising: at least one processor; and memory adapted to store computerprogram instructions, wherein the at least one processor is configuredto: compare a first image and a modified second image, the modifiedsecond image comprising a copy of the first image in which at least onepixel has been replaced, and based on the result of the comparison,generate separate image reconstruction information for each layer ofadditional content which can be combined with the modified second imageto reconstruct the first image.
 14. The apparatus of claim 13, whereinthe at least one processor is further configured to encrypt the separateimage reconstruction information for each layer of the additionalcontent.
 15. The apparatus of claim 13, wherein the separate imagereconstruction information for each layer of the additional contentcomprises at least one residual value each associated with one of the atleast one replaced pixel, and wherein each of the at least one residualvalue is generated by subtracting a pixel value of the at least onereplaced pixel in one of the first image or the second image from apixel value of the at least one replaced pixel in the other one of thefirst image or the second image.
 16. The apparatus of claim 15, whereinthe separate image reconstruction information for each layer of theadditional content comprises a residual image generated by subtractingone of the first image or the second image from the other one of thefirst image or the second image.
 17. The apparatus of claim 16, whereinthe residual image comprises a partial image having a smaller size thaneach of the first image and the second image, wherein the partial imageis generated by subtracting a part of the first image from acorresponding part of the second image.
 18. The apparatus of claim 17,wherein the residual image is divided into a plurality of partial imageregions distributed across the full area of the first image and secondimage, and wherein the residual image comprises at least one partialimage each associated with a corresponding one of the partial imageregions in which at least one pixel has been modified in the secondimage.
 19. The apparatus of claim 17, wherein the separate imagereconstruction information for each layer of the additional contentincludes information identifying a position of the partial image withinthe first image and the second image.
 20. The apparatus of claim 19,wherein the residual image is divided into a plurality of partial imageregions distributed across the full area of the first image and secondimage, and wherein the residual image comprises at least one partialimage each associated with a corresponding one of the partial imageregions in which at least one pixel has been modified in the secondimage.